Profilage et débogage par prise de traces efficaces d'applications hybrides multi-threadées HPC

Supercomputers’ evolution is at the source of both hardware and software challenges. In the quest for the highest computing power, the interdependence in-between simulation components is becoming more and more impacting, requiring new approaches. This thesis is focused on the software development aspect and particularly on the observation of parallel software when being run on several thousand cores. This observation aims at providing developers with the necessary feedback when running a program on an execution substrate which has not been modeled yet because of its complexity. In this purpose, we firstly introduce the development process from a global point of view, before describing developer tools and related work. In a second time, we present our contribution which consists in a trace based profiling and debugging tool and its evolution towards an on-line coupling method which as we will show is more scalable as it overcomes IOs limitations. Our contribution also covers our time-stamp synchronisation algorithm for tracing purposes which relies on a probabilistic approach with quantified error. We also present a tool allowing machine characterisation from the MPI aspect and demonstrate the presence of machine noise for both point to point and collectives, justifying the use of an empirical approach. In summary, this work proposes and motivates an alternative approach to trace based event collection while preserving event granularity and a reduced overheadL’évolution des supercalculateurs est à la source de défis logiciels et architecturaux. Dans la quête de puissance de calcul, l’interdépendance des éléments du processus de simulation devient de plus en plus impactante et requiert de nouvelles approches. Cette thèse se concentre sur le développement logiciel et particulièrement sur l’observation des programmes parallèles s’exécutant sur des milliers de cœurs. Dans ce but, nous décrivons d’abord le processus de développement de manière globale avant de présenter les outils existants et les travaux associés. Dans un second temps, nous détaillons notre contribution qui consiste d’une part en des outils de débogage et profilage par prise de traces, et d’autre part en leur évolution vers un couplage en ligne qui palie les limitations d’entrées–sorties. Notre contribution couvre également la synchronisation des horloges pour la prise de traces avec la présentation d’un algorithme de synchronisation probabiliste dont nous avons quantifié l’erreur. En outre, nous décrivons un outil de caractérisation machine qui couvre l’aspect MPI. Un tel outil met en évidence la présence de bruit aussi bien sur les communications de type point-à-point que de type collective. Enfin, nous proposons et motivons une alternative à la collecte d’événements par prise de traces tout en préservant la granularité des événements et un impact réduit sur les performances, tant sur le volet utilisation CPU que sur les entrées–sortie

MPI Application Binary Interface Standardization

MPI is the most widely used interface for high-performance computing (HPC) workloads. Its success lies in its embrace of libraries and ability to evolve while maintaining backward compatibility for older codes, enabling them to run on new architectures for many years. In this paper, we propose a new level of MPI compatibility: a standard Application Binary Interface (ABI). We review the history of MPI implementation ABIs, identify the constraints from the MPI standard and ISO C, and summarize recent efforts to develop a standard ABI for MPI. We provide the current proposal from the MPI Forum's ABI working group, which has been prototyped both within MPICH and as an independent abstraction layer called Mukautuva. We also list several use cases that would benefit from the definition of an ABI while outlining the remaining constraints

Airway and Esophageal Stenting in Patients with Advanced Esophageal Cancer and Pulmonary Involvement

BACKGROUND: Most inoperable patients with esophageal-advanced cancer (EGC) have a poor prognosis. Esophageal stenting, as part of a palliative therapy management has dramatically improved the quality of live of EGC patients. Airway stenting is generally proposed in case of esophageal stent complication, with a high failure rate. The study was conducted to assess the efficacy and safety of scheduled and non-scheduled airway stenting in case of indicated esophageal stenting for EGC. METHODS AND FINDINGS: The study is an observational study conducted in pulmonary and gastroenterology endoscopy units. Consecutive patients with EGC were referred to endoscopy units. We analyzed the outcome of airway stenting in patients with esophageal stent indication admitted in emergency or with a scheduled intervention. Forty-four patients (58+/-\-8 years of age) with esophageal stenting indication were investigated. Seven patients (group 1) were admitted in emergency due to esophageal stent complication in the airway (4 fistulas, 3 cases with malignant infiltration and compression). Airway stenting failed for 5 patients. Thirty-seven remaining patients had a scheduled stenting procedure (group 2): stent was inserted for 13 patients with tracheal or bronchial malignant infiltration, 12 patients with fistulas, and 12 patients with airway extrinsic compression (preventive indication). Stenting the airway was well tolerated. Life-threatening complications were related to group 1. Overall mean survival was 26+/-10 weeks and was significantly shorter in group 1 (6+/-7.6 weeks) than in group 2 (28+/-11 weeks), p<0.001). Scheduled double stenting significantly improved symptoms (95% at day 7) with a low complication rate (13%), and achieved a specific cancer treatment (84%) in most cases. CONCLUSION: Stenting the airway should always be considered in case of esophageal stent indication. A multidisciplinary approach with initial airway evaluation improved prognosis and decreased airways complications related to esophageal stent. Emergency procedures were rarely efficient in our experience

Profilage et débogage par prise de traces efficaces d'applications hybrides multi-threadées HPC

L’évolution des supercalculateurs est à la source de défis logiciels et architecturaux. Dans la quête de puissance de calcul, l’interdépendance des éléments du processus de simulation devient de plus en plus impactante et requiert de nouvelles approches. Cette thèse se concentre sur le développement logiciel et particulièrement sur l’observation des programmes parallèles s’exécutant sur des milliers de cœurs. Dans ce but, nous décrivons d’abord le processus de développement de manière globale avant de présenter les outils existants et les travaux associés. Dans un second temps, nous détaillons notre contribution qui consiste d’une part en des outils de débogage et profilage par prise de traces, et d’autre part en leur évolution vers un couplage en ligne qui palie les limitations d’entrées–sorties. Notre contribution couvre également la synchronisation des horloges pour la prise de traces avec la présentation d’un algorithme de synchronisation probabiliste dont nous avons quantifié l’erreur. En outre, nous décrivons un outil de caractérisation machine qui couvre l’aspect MPI. Un tel outil met en évidence la présence de bruit aussi bien sur les communications de type point-à-point que de type collective. Enfin, nous proposons et motivons une alternative à la collecte d’événements par prise de traces tout en préservant la granularité des événements et un impact réduit sur les performances, tant sur le volet utilisation CPU que sur les entrées–sortiesSupercomputers’ evolution is at the source of both hardware and software challenges. In the quest for the highest computing power, the interdependence in-between simulation components is becoming more and more impacting, requiring new approaches. This thesis is focused on the software development aspect and particularly on the observation of parallel software when being run on several thousand cores. This observation aims at providing developers with the necessary feedback when running a program on an execution substrate which has not been modeled yet because of its complexity. In this purpose, we firstly introduce the development process from a global point of view, before describing developer tools and related work. In a second time, we present our contribution which consists in a trace based profiling and debugging tool and its evolution towards an on-line coupling method which as we will show is more scalable as it overcomes IOs limitations. Our contribution also covers our time-stamp synchronisation algorithm for tracing purposes which relies on a probabilistic approach with quantified error. We also present a tool allowing machine characterisation from the MPI aspect and demonstrate the presence of machine noise for both point to point and collectives, justifying the use of an empirical approach. In summary, this work proposes and motivates an alternative approach to trace based event collection while preserving event granularity and a reduced overhea

Etude du transport éolien de sable cohésif: application à la dynamique des dunes côtières [Soutenance 01.07.2022]

supervisors Alexandre Valance (department of divided matter) and Pascal Dupontsous la direction de Alexandre Valance dans le département de milieux divisés et de Pascal Dupon

Event Streaming for Online Performance Measurements Reduction

International audienc

A Polyhedral Approach for Auto-Parallelization using a Distributed Virtual Machine

As parallel systems have to undergo an unprecedented transition towards more parallelism and hybridization, we propose to discuss the consequences on programming models. In particular, MPI and OpenMP may face some complexity barriers due to the added complexity required by such hardware. We propose to build from the ground up a new way to program a parallel system, relying both on a distributed runtime, unifying multiple nodes in a coherent ensemble, and on advanced tools from the Polyhedral model. We first describe the Distributed Virtual Machine(DVM) runtime establishing a data-flow environment suitable to the polyhedral transformations.We present and identify what we have seen as key components in such a system, transposing loop nests to a distributed set of machines thanks to a pragma notation combined with an automatic tiling approach. Eventually, while presenting results we open the discussion of remaining challengesand some potential mitigation

Aerodynamic and impact thresholds for cohesive mixture of sand and non-volatile liquid

The moisture has been recognized to have a significant influence on the initiation movement of sand by wind and consequently on sand transport rates. The pertinent literature regarding these phenomena is sparse and current available theoretical and empirical models exhibit considerable disagreement regarding the magnitude of moisture effects. We believe that these discrepancies comes from the fact that the moisture levels are not well controlled neither properly measured and are susceptible to strongly vary over time due to evaporation. To get rid of the variability of moisture content due to evaporation, we propose a new approach based on the use of non-volatile liquid, namely silicon oil instead of water. This insures a proper control of the liquid content and the production of reliable data concerning the variation of the transport threshold with liquid content

Aerodynamic and impact thresholds for cohesive mixture of sand and non-volatile liquid

The moisture has been recognized to have a significant influence on the initiation movement of sand by wind and consequently on sand transport rates. The pertinent literature regarding these phenomena is sparse and current available theoretical and empirical models exhibit considerable disagreement regarding the magnitude of moisture effects. We believe that these discrepancies comes from the fact that the moisture levels are not well controlled neither properly measured and are susceptible to strongly vary over time due to evaporation. To get rid of the variability of moisture content due to evaporation, we propose a new approach based on the use of non-volatile liquid, namely silicon oil instead of water. This insures a proper control of the liquid content and the production of reliable data concerning the variation of the transport threshold with liquid content